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SUMMARY 

In this paper, the recent developments on distributed coordination control, especially the consensus and 
formation control, are summarized with the graph theory playing a central role, in order to present a cohesive 
overview of the multi-agent distributed coordination control, together with brief reviews of some closely 
related issues including rendezvous/alignment, swarming/flocking and containment control. In terms of 
the consensus problem, the recent results on consensus for the agents with different dynamics from first- 
order, second-order to high-order linear and nonlinear dynamics, under different communication conditions, 
such as cases with/without switching communication topology and varying time-delays, are reviewed, in 
which the algebraic graph theory is very useful in the protocol designs, stability proofs and converging 
analysis. In terms of the formation control problem, after reviewing the results of the algebraic graph 
theory employed in the formation control, we mainly pay attention to the developments of the rigid and 
persistent graphs. With the notions of rigidity and persistence, the formation transformation, splitting and 
reconstruction can be completed, and consequently the range-based formation control laws are designed 
with the least required information in order to maintain a formation rigid/persistent. Afterwards, the recent 
results on rendezvous/alignment, swarming/flocking and containment control, which are very closely related 
to consensus and formation control, are briefly introduced, in order to present an integrated view of the graph 
theory used in the coordination control problem. Finally, towards the practical applications, some directions 
possibly deserving investigation in coordination control are raised as well. Copyright © 2013 John Wiley & 
Sons, Ltd. 
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1. INTRODUCTION 

Researchers have long noticed and carried on detailed analysis on many coordinated behaviors, 
for example, the forage for food or defense against predators of insects, birds and fishes in nature, 
and self-organization or self-excitation of particles in physics [1-5]. These drove researchers to 
consider seriously why the creature and particles take initiative to coordinate, and motivated the 
studies and applications on multi-agent coordination. Though it appears to be more complicated 
than single-agent systems, there are indeed many significant advantages in the coordinations of 
MAS (multi-agent system) over the single-agent system, for example [6-111: 

• distributed sensors and actuators, as well as inherent parallelism; 

• larger redundancy, higher robustness and great fault tolerance; 

• performing tasks that single-agent system cannot do; 

• performing usually the tasks that can be finished by single costly agent with lower cost and 
higher excellent performance. 

No doubt that there are more advantages as well. In addition, with the improvements of the 
embedded computing and communicating technologies, and the developments of distributed 
and decentralized methodologies, the distributed coordinations of MASs have become easy to 
materialize. With these inspirations, the problem of distributed coordination control of a network 
of mobile autonomous agents was of interest in control and robotics in the past decade. 
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Generally speaking, distributed coordination control uses local interactions between agents to 
achieve collective behaviors of multiple agents, and therefore to accomplish global tasks. It has a 
broad range of potential applications, such as in military, aerospace, industry, and entertainment. 
Figure 1 shows some typical applications of multi-agent distributed coordinations. In the military 
field, multiple mobile robot systems can adopt a proper geometric pattern to perform military tasks 
for taking the place of human soldiers, such as reconnaissance, searching, mine clearance, and 
patrol under adverse/hazardous circumstances. Taking the reconnaissance mission as an example, 
a single robot has limited ability to gain environmental information, however, if multiple robots 
keep proper formation to cooperatively apperceive the surrounding, they are likely to rapidly and 
accurately obtain the environmental information. In the aerospace field, satellite formation is the 
leading technique in the space application in 21 th century, which opens up a brand-new direction 
for the application of satellites, especially for mini-satellites. Satellite formation cannot only greatly 
reduce the cost and enhance the reliability and survivability, but also broaden and override the 
function of individual satellites and achieve the tasks that multiple single spacecrafts cannot finish. 
In the industrial field, multiple mobile robots can deal with some dull, dirty and dangerous work in 
formation. For example, when multiple robots cooperatively carry large scale goods in a poisonous 
environment, their positions and orientations are strictly restricted in order to meet the requirements 
of load balance. In the entertainment field, for example, multiple dancing robots or soccer robots, 
in order to keep neat or meet tactical needs, must keep some special patterns, and may dynamically 
switch the patterns for avoiding obstacles. 

Distributed coordination control of MASs has attracted tremendous attention with various aspects 
including consensus, formation, rendezvous, alignment, synchronization, swarming, flocking, 
containment control, gossip, cooperative searching and reconnaissance, cooperative location and 
mapping, to name a few. A number of methods have been proposed for achieving distributed 
coordination control of MAsS, and conventional methods include the leader-follower method [12- 
lb], behavior-based method [17-20], virtual structure method [21-24], graph-based method [6, 
7, 25-27], etc. But now, these methods are gradually mixed together, and it is hard to clearly 
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Figure 1. Some typical applications of multi-agent distributed coordinations 


distinguish from each other. In particular, the graph-based method has become the dominator, 
because MASs can be modeled by a graph naturally, and almost all the aspects of coordination 
control can be then studied by utilizing the graph theory. Therefore, this paper mainly focuses on 
the recent developments of consensus, formation control, and some closely related topics such as 
rendezvous/alignment, swarming/flocking and containment control, with the graph theory paying 
central roles, in order to present a cohesive overview of the multi-agent distributed coordination 
control. Also, some topics which might be interesting for future investigation are discussed. 


2. RECENT DEVELOPMENTS 

A MAS is in general distinguished with larger scale (often decided by the numbers of agents) and 
decentralized perception, communication and control structures, and forms inter-connected network 
between agents consequently. Thus, it can be naturally modeled by a graph with vertices being used 
to describe agents and the edges being used to represent topological relationships between agents 
such as perception, communication and control links. Both directed and undirected graphs can be 
used to describe the MAS. For example, when the i th agent is required to keep the predetermined 
distance to the j th agent, while the j th agent is not required to keep the distance to the i th agent, 
the edge ij is directed from vertex i to j, and at this time a directed graph is used to model the 
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MAS. Conversely, when the interconnection between agents i th and j th is bi-directional, when the 
i th agent could perceive the j th agent if and only if the j th agent could perceive the i th agent for 
example, the edge ij is undirected, and at this time the graph is also undirected. 

After modelled by a graph, as noted above, the mature graph theory can be borrowed to study the 
coordination control of MASs. In recent years, many monographs about the research results of the 
graph theory on multi-agent coordination control problems, e.g. [8-11], are published. Therefore, 
in the sequel, we will mainly pay attention on the survey of recent developments on some typical 
multi-agent coordination problems, including consensus, formation control, rendezvous/alignment, 
swarming/flocking and containment control, with graph theory playing a central role. The 
relationships among the investigated coordination control problems are summarized as Fig. 2. 



Figure 2. Relationships among consensus, formation control and some closely related topics, i.e., 
rendezvous, alignment, swarming, flocking and containment control. 


2.1. Consensus 

In the networks of agents, the consensus means to reach an agreement regarding a certain quantity 
of interest that depends on the states of all agents. A consensus protocol is an interaction rule that 
specifies the information exchange between an agent and all of its neighbors in the network [28,29]. 

In general, there are two key elements considered in consensus problems, that is, the dynamics 
of agents and the communications among agents, as illustrated in Fig. 3. To be more precise, two 
parts are normally taken into account in the agents’ dynamics, which are the dynamics of the agent 
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itself and the protocol used among agents. The former is the inherent model of the agents, such 
as the first/second/high-order models, linear/nonlinear models, and continuous/discrete models; the 
latter is used to modify the agent inherent model in order to make the MAS achieve a consensus. 
The communications among agents can also be divided into communicating structures such as the 
shapes and changing of the communication topology, and data transmissions including time delays 
and transmission failure. 



Dynamics- 


Agent (Inherent Models) 
Protocols (Modified Models) 


Communications— 


Structures 

Data Transmissions 


Figure 3. Key elements considered in consensus problems 


On the other hand, the consensus problems can also be divided into two categories by considering 
the predominance of agents. If an agent has priority so that the other agents must follow its trajectory, 
then such an agent is called the “leader”; accordingly, if the MAS has at least one leader, then the 
consensus is leader-follower consensus; otherwise, it is leaderless consensus. Even though it seems 
there is an obvious difference between leader-follower and leaderless consensus, they have the same 
foundations, because the topology and consensus protocol can make a common agent act as a leader. 
Afterwards, we will mainly focus on the developments of leaderless consensus and then point out 
some important results about leader-follower consensus. 

The beginning of researches on consensus is [30], in which Vicsek et al. propose a simple discrete¬ 
time model of particles all moving in the plane at the same speed but with different headings. 
Afterwards, the algebraic graph theory, which studies the relationships between the structure of 
graphs and different matrix representations of graphs, is widely employed in the consensus [31]. The 
most important concepts of the algebraic graph theory used in MASs are the adjacent matrix and 
the Laplacian matrix. For a graph Q = (V, £) consisting of a nonempty vertex set V = A ? } 

and an edge set £ C V x V, the adjacency matrix A[Q) = \a, :i } is an N x N matrix given by 
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aij = 1, ciij > 0 if and only if (i,j) e E, and a,y = 0, otherwise. The degree of vertex i is given by 
di = ^2 j a^, and the Laplacian matrix of Q is L n (Q) = diag(di, ..., djy) — The pioneering 
work to deal with consensus by virtue of the Laplacian matrix is literature [18], in which a theoretical 
explanation for the observed behavior in [30] is provided. After that a large number of literatures 
use the algebraic graph theory to study the consensus problem, for example, literature [29, 32-40], 
to name a few. 

The most basic case of consensus is that the considered agents are governed by linear dynamics, 
especially by single-integrator dynamics. The theoretical framework for reaching a consensus for 
networked agents with single-integrator dynamics under fixed/switching communicating topology 
was introduced by Olfati-Saber and Murray in literature [32], building on their earlier work of [25]. 
The subsequent studies mainly followed up on the way of [32]. For single-integrator dynamics, the 
commonly used continuous-time consensus protocol is [29, 32-34, 39] 

n 

X,(t) = -^2 a ij(t)(xj(t) - Xi(t)), i = 1,. .. ,n; (1) 

. 7=1 

for first-order discrete systems, the commonly used discrete-time consensus protocol is [18,29,34, 
40] 

n 

Xi[k + 1] = y^ffljj[fc]a;j[A;], i = l,...,n, (2) 

o =i 

in which Xi(t) and Xi[k] represent the states of the i th agent, a,j (f) and a. VJ [k] are the (i,j) entry 
of the adjacent matrix of the associated communication graph of MASs at time t or k(t). Note that 
protocols (1) and (2) only use the relative state information of the adjacent agent, so it is local and 
distributed. Then, by using the tools of the algebraic graph theory, different conclusions for different 
cases are made. Under the condition of the fixed topology, protocols ( 1 ) and (2) can guarantee a 
consensus if the communication graph is connected for an undirected graph or has a rooted spanning 
tree for a directed graph. By contrast, when the topology is switching, protocols ( 1 ) and (2) result 
in a consensus if there exists an infinite sequence of contiguous, uniformly bounded time intervals, 
with the property that across each interval, the union of the communication graphs is connected 
for an undirected case or has a rooted directed spanning tree for a directed case. Furthermore, by 
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using the maximum and minimum eigenvalues of the Laplacian matrix, the converging rate of the 
consensus protocol is analyzed [41]. 

Time delays associated with both message transmission and processing, for instance caused by 
the communication congestion, are inevitable and also taken into account in consensus problems. 
Let Sij denote the time delay communicated from the j th agent to the i th agent, protocol (1) is 
modified as 

n 

i'i (f) — ^ ' dij (f) (xj (f Sij ) Xi (f Sij )) • (3) 

4=1 

Reference [32] investigated the simplest case where Sij = S and the communication topology was 
fixed, undirected, and connected, and concluded that the consensus was achieved if and only if 
0 < 5 < 7r/(2A lnax (/.)), where L denoted the Laplacian of the communication graph. Along the 
way developed by [32], different time delays and communication topologies are considered in the 
consensus problem. If the time delay affects only the state being transmitted, protocol (3) is modified 
as 

n 

±i(t) = - - Sij) -Xi(t)), (4) 

4=1 

and the consensus for switching directed topologies remains valid for an arbitrary time delay when 
= S [42], The constant or time-varying, uniformly or non-uniformly distributed time delays 
are considered in [43], and sufficient conditions for the existence of average consensus under 
bounded communication delays are given correspondingly. The consensus problem with noise- 
perturbed under fixed and switching topologies as well as time-varying communication delays 
is investigated in [44]. It is shown that the consensus is reached for arbitrarily large constant, 
time-varying, or distributed delays if consensus is reached without delays [45]. For the discrete 
consensus protocol (2), conclusions similar to that of the continuous case can be obtained. It is 
shown in [46] that if a consensus is reached under a time-invariant undirected communication 
topology, then the presence of communication delays does not affect the consensus. In addition, 
sufficient conditions for the consensus under dynamically changing communication topologies and 
bounded time-varying communication delays are shown in reference [47]. 
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The results of the first-order system have been extended to the consensus of the second-order 
and high-order linear dynamical MASs, and similar consensus protocols and graph conditions are 
obtained. Consensus for multiple agents governed by the second-order linear dynamics has been 
considered by the similar framework of using the algebra graph theory [48-511. For example, in 
the representative work [48], a linear distributed consensus protocol for the second-order MAS 
is designed with aid of the Laplacian matrix without requiring velocity information of neighbors; 
a variable structure control law is used to design the consensus protocol by taking the second- 
order system as two cascade fist-order systems in [51]. In terms of high-order consensus problems, 
the first discussion is completed by W. Ren in [52], where the second-order MAS is generalized 
to the I th -order integrator MAS. Afterwards, the consensus protocol is modified in [53] and y- 
consensus is investigated under undirected communication topology. It should be noted that both 
the aforementioned literatures have no self-feedback information in the consensus protocols. With 
adding the self-feedback items, reference [54] investigates the constant-value consensus problem 
for high-order MASs under fixed and switching directed topology. Based on those meaningful 
explorations, the robust analysis and the converging results with time delays are further provided, 
for example, the work in [55-57]. The consensus governed by more complex high-order linear 
dynamics, such as the SISO (single input single output) and the MIMO (multiple inputs multiple 
outputs) linear dynamics are also investigated. At the beginning, researcher did not notice the 
relationship between the consensus in MAS and the controllability in individual agents; therefore, 
many studies are done with the assumption that the isolated agent is controllable, e.g. the work in 
[58]. It is proven that this technical assumption is not necessary in [59], excluding the necessity 
of a sufficient connection of the graph topology is required for consensus stability for a system 
with unstable agent dynamics. Afterwards, the studies on communication delays and robustness are 
conducted, e.g. [60,61], And further, a necessary and sufficient condition for high-order consensus 
with unknown communication delays is given for the existence of solution to heterogeneous multi¬ 
agent systems in [62]. 
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Considering that most physical systems are inherently nonlinear in nature, the consensus where 
the agents are governed by nonlinear dynamics has also aroused the attention of some researchers 
recently. It can be shown, for example, in [63,64] where the continuously differentiable nonlinear 
dynamics are considered; in [65-67] where the nonlinear dynamics satisfy the global Lipschitz 
condition under a directed communicating graph, respectively, without and with a leader agent; and 
in [68] where an adaptive control method is introduced to study the synchronization of uncertain 
nonlinear networked systems. 

2.2. Formation Control 

The, formation control is that a team comprised of multiple agents keeps a predetermined geometric 
pattern and adapts to the environmental constraints (e.g. obstacle avoidance) at the same time during 
the movement towards a specific goal. In general, the main control problems in formation are 
summarized as follows [69,70]: 

• Formation generation: how to design a formation pattern for MASs and then achieve it? 

• Formation maintaining: how to maintain the formation pattern for MASs during the 
movement? 

• Formation transformation: how to transit the formation pattern, which means transiting one 
formation pattern into another? 

• Obstacle avoidance with formation: how to change a motion plan or a formation pattern for 
MASs in order to avoid obstacles during movement? 

• Self-adaptation: how to automatically change the formation in order to best adapt to the 
dynamical unknown environment? 

The graph theory is a powerful tool to study formation control involving all the above five 
aspects. Firstly, graphs are naturally used to generate and maintain a formation, and consequently 
to achieve transformation between different formation patterns. In early 2001, directed graphs 
were used to describe the topologies and patterns of formation, and then study the formation 
evolution problem [12]. Saber et al adopted the combination graph theory to obtain a unique 
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determined formation pattern through designing the weights of edges [71]. Desai et al in [13] 
further adopted directed acyclic graphs to represent the control graph between agents, and 
designed the control strategy, afterwards, through enumerating all possible control graphs to realize 
transformation between any two formation patterns. It is worthy pointing out that by adding a 
pre-specified geometric pattern, the formation generation and maintaining can also be achieved 
in the corresponding consensus problem, which is usually known as consensus-based formation 
control, e.g., in [72,73]. Consequentially, the algebra graph theory is widely employed in formation 
control too. As aforementioned, the neighbors of vertex i in the graph represented MAS is exactly 
the collection of agents that have a topological relationship such as perception with the i th agent. 
Therefore, by using the properties of the Laplacian matrix, local, distributed and scalable formation 
controllers can be designed, and their stabilities can also be verified by virtue of the eigenvalue of 
the Laplacian matrix, for a typical example, the work in the classic literature [26]. And then, Fax 
and Murray set up the relation between the formation controller and the topological structure of the 
communicating network with the Laplacian matrix, and proved that if the local controller was stable, 
then the stability of formation with linear dynamics depended on the stability of the information 
flow [25]; Lin et. al proved that if and only if there is a global accessible vertex in the perception 
graph, the formation is stable by using tools of the algebraic graph theory [74]. In addition, the 
formation stability concepts are defined with the aid of the graph theory. Tanner et. al defined the 
leader-to-formation stability notion based on the graph and input-to-state stability concept, which 
includes both transient and steady-state errors [75]. In contrast, the pairwise asymptotic stability is 
defined based on directed graphs by involving a single pair of neighboring agents in [76], which 
implies that any two agents can asymptotically achieve and maintain a desired relative attitude and 
position though the pair of agents which may not be neighbors and do not interact with each other 
directly. 

Another important tool of the graph theory employed in multi-agent formation control is the 
rigid graph theory. An undirected graph is rigid if for almost every structure, the only possible 
continuous moves are those which preserve every inter-agent distance; further, an undirected graph 
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is called minimally rigid if it is rigid and if there exists no rigid graphs with the same number of 
vertices and a smaller number of edges (see Fig. 4 for more illustrations of rigid and minimally 
rigid graphs). The main tools of rigid graphs are Henneberg sequence and Laman’s theorems [77]. 





Figure 4. (a) rigid graph (b) minimally rigid graph (c) non-rigid graph 


The former was raised by Henneberg to construct two-dimensional minimally rigid graphs, and the 
latter were raised by Laman in 1970 to verify if a two-dimensional graph was rigid [78]. With 
the concepts of rigid graphs, some primitive operators are defined to deal with the operations 
such as splitting and restructuring of rigid formation [71,79]; also, a formation control law is 
proposed by only using the distances between agents in order to ensure rigidity or generic rigidity 
of formation [25]. Note that the concept of rigidity is mainly for the undirected graph, which 
means the interrelations between vertices are bi-directional. However, in practical applications, the 
MAS is often modeled by a directed graph for reducing the cost of communication and perception. 
Hence on the basis of rigidity, a group led by Prof. Brian D.O. Anderson at the Australian National 
University developed a “directed rigidity” concept, named persistence. A directed graph is persistent 
if and only if its underlying graph is rigid, and itself is constraint consistent [27], as illustrated in 
Fig. 5 t. Similar to a minimally rigid one, a graph is minimally persistent if it is persistent and if 
no single edge can be removed without losing persistence. The main difference between rigidity 
and persistence is that rigidity assumes all the constraints are satisfied, as if they were enforced 
by an external agency or through some mechanical properties, while persistence considers each 

t In R 2 , the graph represented in (a) is not persistent. For almost all uncoordinated displacements of 2, 3 and 4 (even 
if they satisfy their constraints), 4 is indeed unable to satisfy its three constraints. This problem cannot happen for the 
graph represented in (b), which is persistent however. 
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constraint to be the responsibility of a single agent. Based on the concept of persistence, many 




Figure 5. (a) nonpersistent graph in R 2 (b) persistent graph in R 2 


works have been investigated by the group led by Prof. Brian D.O. Anderson, such as developing 
the persistent/minimum persistent notions from the rigid/minimum rigid concepts [27]; fixing 
the transformation, splitting and reconstruction of formation in a two-dimensional space through 
defining three primitive graph operators by virtue of Henneberg sequence [80, 81]; designing a 
formation control law to keep the persistence for formation in two-dimensional space [82]; and 
further proposing distributed formation control laws in order to keep the minimum persistence for 
two kinds (leader-remote-follower and coleader) of formation with small perturbations [83]. It is 
worthy pointing out that the persistence notions have been applied to practical multiple robots 
systems. For example, B. Smith et al designed an automatic generation algorithm of persistent 
formations for multiple robots under distance and communication constraints [84], and a method for 
accomplishing such persistent formations was presented and further demonstrated with a prototype 
network of robots in a NASA project for researches in Antarctica [85]. 

2.3. Some closely related issues 

2.3.1. Rendezvous/Alignment The rendezvous problem refers to designing a distributed local 
control strategy to make multiple agents reach the same specified position at the same time. To some 
extent, rendezvous can be taken as the application of consensus in actual systems, such as robots 
and spacecrafts [86]. This problem was firstly proposed in literature [87], in which a distributed 
simple memoryless point rendezvous algorithm was proposed with proven convergence. And then, 
literature [88] and [89] extended this algorithm to a “stop-and-go” strategy under synchronous 
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and asynchronous situations, respectively, and analyzed the convergence of the strategies. Similar 
to other multi-agent coordinations, the graph theory also played key roles in the research of 
the rendezvous problem. Cortes et. al presented a robust rendezvous algorithm in an arbitrarily 
dimensional space with the aid of the proximity graph [90]; and a connectivity-preserving protocol 
consisting of a set of distributed control rules was proposed in the case of the link in communication 
graph failure in literature [91]. In addition, in view of different kinds of practical systems, the 
rendezvous problem has received extensive researches. For multiple omni-directional mobile robots 
equipped with line-of-sight limited-range sensors moving in a connected, nonconvex, unknown 
environment, literature [92] presented a perimeter minimizing rendezvous algorithm by using the 
notions of connectivity-preserving constraint sets and proximity graphs. For multiple nonholonomic 
unicycles, a discontinuous and time-invariant rendezvous algorithm was designed with tools from 
the nonsmooth Lyapunov theory and the graph theory [93]. The discontinuous rendezvous policies 
were further investigated in literature [94], in which the new proposed sufficient conditions for 
characterizing the control policies were less restrictive than those presented in the above-mentioned 
literature. For a multiple agent system moving like a Dubins car, a simple quantized control law was 
designed with three values to make agents achieve rendezvous given a connected initial assignment 
with minimalism in sensing and control [95]. 

The attitude alignment problem, sometimes called attitude consensus or attitude synchronization 
problem, also received extensive attention in multi-agent fields, especially multiple spacecrafts [96], 
multiple satellites [97] and multiple marine robots [98]. Similar to the rendezvous problem, 
the attitude alignment is required to design a distributed control strategy to make the attitude of 
multiple agents tend to be consistent simultaneously. Lawton et al [99] adopted the behavior-based 
method to design two kinds of control strategies that made a group of aircrafts achieve attitude 
alignment; and further, this work was extended to more general scenes which did not need bi¬ 
directional communication [96]. For the attitude alignment with limited communication and no 
reference signal, Sarlette et al proposed attitude alignment strategies based on artificial potential 
methods [100]. 


hit. J. Robust. Nonlinear Control (2013) 


Copyright © 2013 John Wiley & Sons, Ltd. 
Prepared using rncauth.cls 


DOI: 10.1002/mc 


MULTI-AGENT DISTRIBUTED COORDINATION CONTROL: DEVELOPMENTS AND DIRECTIONS 15 


2.3.2. Swarming/Flocking The swarming is often inspired by biological and physical systems, 
which refers to the prevalent collective behavior and the self-organization phenomenon, such 
as ant colonies, bee colonies, flocks of birds and schools of fishes [2-4]. And recently it has 
emerged in MAS with focuses on physical embodiment and realistic interactions among the 
individuals themselves and also between the individuals and the environment [101]. Generally 
speaking, swarming is characterized by the following: 1) flexibility: adaptability to the environment; 
2) robustness: anti-jamming to the internal and external disturbances; 3) dispersion: dynamical 
behavior based on individuals; 4) self organization: obvious overall system properties in evolution, 
namely emerging [102]. 

As a special case of swarming, the flocking refers to the phenomena that the MAS (usually 
with the second-order integrator dynamics) presents certain coordinated behaviors by decentralized 
control with the aid of local perception and reaction between individuals. The classical flocking 
model is proposed by Reynolds in [103], in which the animation of flocking behaviors, such as bird 
flock and fish school, are realized through three rules, i.e. collision avoidance, velocity matching and 
flock centering. These three rules can make the distance between agents converge to an expected 
value, the speed of agents tend to be consistent, and agents cannot collide with each other. After then, 
for the linear systems with the second-order integrator dynamics, flocking algorithms are usually 
designed by local artificial potential fields integrated with the graph theory [104-106]. In [104], 
a theoretical framework based on the algebraic graph theory and the spatially induced graphs is 
provided for the design and analysis of scalable flocking algorithms under a connected topology. 
Further, this work was extended in [105] from two directions: one is the case where only a fraction 
of agents are informed and the other is that where the velocity of the virtual leader is varying. 
In [106], the stability properties of a group of agents governed by decentralized, nearest-neighbor 
interaction rules are analyzed. The flocking problem without considering collision between multi- 
agents has also got extensive attentions in recent years, e.g. in [107,108]. While little work has been 
done on flocking with nonlinear dynamics, except that Dong proposed a backsteping based control 
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law design method for the flocking of multiple nonholonomic wheeled mobile robots with directed 
topology in [109]. 


2.3.3. Containment control The containment control can be taken as an expansion of consensus 
or flocking with pinning control, which refer to designing a distributed control law to drive the 
states of the followers to the convex hull spanned by the states of multiple leaders. In particular, 
when the multiple leaders are moving, some literature call it as set tracking problem [110]. In 
early 2005, the simplest containment control, which took two agents as leaders and drove the rest 
converge to the line decided by the two leaders, was discussed in [26]. For the containment control 
with the first-order dynamics under fixed undirected graphs, a simple “stop-and-go” strategy is 
proposed and analyzed in literature [111]. Further, the containment control with single- or double¬ 
integrator dynamics was studied in [111-1 15], of which, literature [114] proposed two containment 
control algorithms via only position measurements where the leaders were neighbors of only a 
subset of the followers, and literature [115] examined the containment control for a second-order 
multi-agent system with random switching topologies. Moreover, the containment control strategy 
for multiple Lagrangian system in directed topological structure was proposed in [116]; and the 
containment control of multiple rigid-body systems with uncertainty was studied in [117]. For the 
more general nonlinear dynamics, G. Shi et al [ 110] investigated the distributed set tracking problem 
with unmeasurable velocities under switching directed topologies, and provided the necessary and 
sufficient conditions for set input-to-state stability and set integral input-to-state stability. 

Remark 1 

The studies on coordination control are still ongoing, both in depth and width. On the one hand, 
the studied agents are found with more and more complicated dynamics and limited perceptions, 
for example, from high-order dynamics [62] to complex environmental constraints [118], again 
to communication bandwidth limitation [119-121], and quantitative communication [122]. On 
the other hand, more and more attention has been paid to various practical multi-robot systems, 
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for example, multiple micro-satellites [123], multiple spacecrafts [124, 125], multiple marine 
robots [126], multiple wheeled mobile robots [127,128], and multiple Lagrangian systems [129]. 


3. DIRECTIONS 

The theoretical work on multi-agent coordination control has made significant achievements in the 
past decade, which have however rare practical applications. In fact, the behaviors of actual robots 
are much more complicated than those considered in the existing work. Therefore, how to promote 
the theoretical work for serving in practice in order to shrink the gap between theory and practice 
is still an open problem which should be considered seriously. In the authors’ opinion, towards 
this direction, there are several topics deserving further investigation in the study of multi-agent 
coordination control, which listed as follows: 

1) Coordination control with strongly nonlinear dynamics 

The existing coordination control often considers the agents governed by linear dynamics, 
especially by the single- or double-integrator dynamics; however, most physical systems are 
inherently nonlinear in nature. At present, coordination control with nonlinear dynamics has 
received seldom consideration, and few related work assume the nonlinear dynamics to be 
continuously differentiable [63, 64] or globally Lipschitz [65, 66]. In contrast, the dynamics of 
robots, vehicles or UAVs in practice are generally found with strong nonlinearities, which are 
hard to be described simply by using single/double-integrator or continuously differentiable/global 
Lipschitz continuous function. So it is necessary and beneficial to study multi-agent coordination 
control in the presence of the nonlinearities by means of the nonlinear control theory. 
Literature [ 130,131 ] carried on some preliminary research on this way via the input-to-state stability 
concept, but more problems, such as the finite time convergence, converging rate analysis, with 
topological switching and time-delay, are still worthy of further exploration. 

2) Coordination control in three-dimensional space 

The existing work about multi-agent coordinations usually considers agents in a plane; while 
the coordinations in the more general three-dimensional space, composed of three rotational and 
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three translational degrees of freedoms, are not extensively investigated. In essence, any general 
rigid motion (including rotation and translation) is described in three-dimensional space and its 
subspaces, and the coordination control in a plane is just a special case in three-dimensional space. 
Moreover, the applications of multi-agent coordination control are not confined to be in the plane. 
For instance, in the applications of multiple spacecraft formation flights [123], multiple marine 
robots cooperative exploration [126], flight array [132], and multiple flying vehicles cooperative 
handling objects [124] or playing tennis [125], the coordination controls are required to be 
considered in the three-dimensional space with attitude and position control simultaneously. Such 
problems are worthwhile to be solved, in the authors’ opinion, for coordination control in three- 
dimensional space include two aspects: a) the graph theory in three-dimensional space. As noted 
above, the graph theory is an important tool used in multi-agent coordination control, however, the 
existing results on the graph theory cannot be directly extended to the three-dimensional space. 
For example, Laman's theorem and Henneberg sequence in three-dimensional space are still open 
problems [82]. Improving and expanding the existing foundations of the graph theory in order to 
make them valid in three-dimensional space is still a problem that has not been effectively solved, 
b) designing the coordination control law in a three-dimensional space. In recent years, different 
mathematical tools have been used to study the multi-agent coordination control in the three- 
dimensional space, such as the decoupling method (namely carrying on the attitude control and 
the position control independently) [123,125], the Lie-group abstraction [133], the homogeneous 
transformation matrix [134,135], and the dual quaternion approach [76], etc. Whatever, the above 
methods are still not systematic, and it is necessary to combine with the graph theory to further 
design coordination control laws under different topology structures, with different dynamic models 
and adaptation to external disturbances. 

3) Coordination control with practical constraints 

In practice, there are many constraints for the coordinations of MASs, such as non-ideal 
communications and perceptions, and heterogeneous dynamics. Challenges from pure theories to 
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practical applications should be attached with more extensive concern, in order to shrink the gap 
between theories and applications. 

Communications are very important for multi-agent coordinations. Communications for actual 
systems are usually not ideal.For example, time-delays, unstable signals, and limited bandwidth 
are inevitable in the actual systems. Therefore, multi-agent coordination control with non-ideal 
communication is a challenging problem, and many work can be further considered, such as the 
design of fault tolerance topological structure which is robustness against the loss of communicating 
nodes or links, coordination algorithms with limited communicating bandwidth or quantitative 
communication, or even without communications, and underlying communication protocols with 
reliable robustness. 

In addition, it is usually the ideal perceptions that is considered by the existing MASs, which 
means agents can obtain all the required information in real-time. However in practice, sensors 
equipped by agents certainly have some perceptional limitations. For example, the commonly used 
cameras have conical perceptional fields, and the laser ranging sensor (typically the Hokuyo’s series 
products) can only perceive the information in a sector. Meanwhile, the perceived information 
is usually accompanied by noise and time-delay, and further more information, such as other 
agents’ relative velocities and accelerations, is generally difficult to obtain precisely. Therefore, 
the coordination control with perceptional constraints is also a problem that requires to be fixed, 
such as coordination control with perceptional directions, coordination control without velocity 
measurement, and coordination control with time-delay and noisy information. 

Thirdly, in the existing work on coordination control, the agents are usually considered to be 
isomorphic, which means all agents are found with the same dynamics. However, in many practical 
applications, for example, in the cooperative reconnaissance carried out by manned/unmanned aerial 
vehicles, and the cluster spacecraft system, the dynamics between agents are quite different. So, the 
coordination control for heterogeneous MASs is also a direction worth to pay attentions to in the 
future. 
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Finally, most of the existing theoretical results are only verified by simulations, rather than by 
actual systems, partly due to the high cost and various restrictions of the experiments. Therefore, to 
verify and apply the theoretical results to actual multi-robot systems is a most pressing issue too. 

4) Combination with other collective behaviours 

For the last ten years, the work on coordination control in fact has led the research of the 
distributed networked system and provided some necessary supports to different types of collective 
behaviors, for e.g. the wireless sensor network. Therefore, how to expand and fuse the existing 
results of coordination control to other collective behaviours such as wireless sensor network is also 
a notable direction. 


4. CONCLUSION 

This paper reviews the recent developments on multi-agent coordination control with focus on 
the consensus, formation control, and some closely related topics including rendezvous/alignment, 
swarming/flocking and containment control, with the graph theory playing a central role, in order 
to present a cohesive overview of the multi-agent distributed coordination control, and further 
provides some directions possibly deserving to be investigated in coordination control. The work 
on coordination control involves extremely extensive contents, and the authors believe that in 
the coming decade, more problems related to coordination control will be solved along with 
the developments of other related disciplines and technologies, and will also get more practical 
applications in more fields. 
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